Large scale log file processing systems typically keep track of a huge number of growing log files, generate work units (e.g., bundles) from newly written log file chunks, and process the bundles to produce results. It is desirable to process each bundle exactly once to avoid duplicative processing. However, it is also desirable to implement multiple processing pipelines to improve the robustness of the system and to assure a minimum level of fault tolerance (e.g., to provide protection against pipeline failures/crashes). It is difficult and challenging to implement a multi-site log file processing system while ensuring minimal processing redundancy.